Accuracy and Reliability of Chatbot Responses to Physician Questions.

Importance: Natural language processing tools, such as ChatGPT (generative pretrained transformer, hereafter referred to as chatbot), have the potential to radically enhance the accessibility of medical information for health professionals and patients. Assessing the safety and efficacy of these tools in answering physician-generated questions is critical to determining their suitability in clinical settings, facilitating complex decision-making, and optimizing health care efficiency. Objective: To assess the accuracy and comprehensiveness of chatbot-generated responses to physician-developed medical queries, highlighting the reliability and limitations of artificial intelligence-generated medical information. Design, Setting, and Participants: Thirty-three physicians across 17 specialties generated 284 medical questions that they subjectively classified as easy, medium, or hard with either binary (yes or no) or descriptive answers. The physicians then graded the chatbot-generated answers to these questions for accuracy (6-point Likert scale with 1 being completely incorrect and 6 being completely correct) and completeness (3-point Likert scale, with 1 being incomplete and 3 being complete plus additional context). Scores were summarized with descriptive statistics and compared using the Mann-Whitney U test or the Kruskal-Wallis test. The study (including data analysis) was conducted from January to May 2023. Main Outcomes and Measures: Accuracy, completeness, and consistency over time and between 2 different versions (GPT-3.5 and GPT-4) of chatbot-generated medical responses. Results: Across all questions (n = 284) generated by 33 physicians (31 faculty members and 2 recent graduates from residency or fellowship programs) across 17 specialties, the median accuracy score was 5.5 (IQR, 4.0-6.0) (between almost completely and complete correct) with a mean (SD) score of 4.8 (1.6) (between mostly and almost completely correct). The median completeness score was 3.0 (IQR, 2.0-3.0) (complete and comprehensive) with a mean (SD) score of 2.5 (0.7). For questions rated easy, medium, and hard, the median accuracy scores were 6.0 (IQR, 5.0-6.0), 5.5 (IQR, 5.0-6.0), and 5.0 (IQR, 4.0-6.0), respectively (mean [SD] scores were 5.0 [1.5], 4.7 [1.7], and 4.6 [1.6], respectively; P = .05). Accuracy scores for binary and descriptive questions were similar (median score, 6.0 [IQR, 4.0-6.0] vs 5.0 [IQR, 3.4-6.0]; mean [SD] score, 4.9 [1.6] vs 4.7 [1.6]; P = .07). Of 36 questions with scores of 1.0 to 2.0, 34 were requeried or regraded 8 to 17 days later with substantial improvement (median score 2.0 [IQR, 1.0-3.0] vs 4.0 [IQR, 2.0-5.3]; P < .01). A subset of questions, regardless of initial scores (version 3.5), were regenerated and rescored using version 4 with improvement (mean accuracy [SD] score, 5.2 [1.5] vs 5.7 [0.8]; median score, 6.0 [IQR, 5.0-6.0] for original and 6.0 [IQR, 6.0-6.0] for rescored; P = .002). Conclusions and Relevance: In this cross-sectional study, chatbot generated largely accurate information to diverse medical queries as judged by academic physician specialists with improvement over time, although it had important limitations. Further research and model development are needed to correct inaccuracies and for validation.

New Techniques for Intraoperative Parathyroid Localization.

Accurate identification of abnormal parathyroid glands (PGs) during parathyroidectomy and thyroidectomy can be challenging even for experienced surgeons given PGs variable location, size, and similar appearance to surrounding tissue. Inadvertent removal or devascularization of healthy PGs can lead to transient or permanent hypoparathyroidism. Permanent hypoparathyroidism is associated with increased rates of renal insufficiency, seizures, skeletal abnormalities, increased costs, decreased quality of life, and increased mortality. Conversely, the inability to identify and remove hyperfunctioning PGs results in failed parathyroidectomy which can result in need for reoperations that are associated with increased technical difficulty, operative duration, rates of hypoparathyroidism and recurrent laryngeal nerve damage, and cost.

Does the Use of Probe-based Near-infrared Autofluorescence Parathyroid Detection Benefit Parathyroidectomy?: A Randomized Single-center Clinical Trial.

OBJECTIVE: To evaluate the benefits of probe-based near-infrared autofluorescence (NIRAF) parathyroid identification during parathyroidectomy. BACKGROUND: Intraoperative parathyroid gland identification during parathyroidectomy can be challenging, while additionally requiring costly frozen sections. Earlier studies have established NIRAF detection as a reliable intraoperative adjunct for parathyroid identification. METHODS: Patients undergoing parathyroidectomy for primary hyperparathyroidism were prospectively enrolled by a senior surgeon (>20 years experience) and a junior surgeon (<5 years experience), while being randomly allocated to the probe-based NIRAF or control group. Data collected included procedure type, number of parathyroids identified with high confidence by the surgeon and the resident, number of frozen sections performed, parathyroidectomy duration, and number of patients with persistent disease at the first postoperative visit. RESULTS: One hundred sixty patients were randomly enrolled under both surgeons to the probe group (n=80) versus control (n=80). In the probe group, parathyroid identification rate of the senior surgeon improved significantly from 3.2 to 3.6 parathyroids per patient ( P <0.001), while that of the junior surgeon also rose significantly from 2.2 to 2.5 parathyroids per patient ( P =0.001). Parathyroid identification was even more prominent for residents increasing significantly from 0.9 to 2.9 parathyroids per patient ( P <0.001). Furthermore, there was a significant reduction in frozen sections utilized in the probe group versus control (17 vs 47, P =0.005). CONCLUSION: Probe-based NIRAF detection can be a valuable intraoperative adjunct and educational tool for improving confidence in parathyroid gland identification, while potentially reducing the number of frozen sections required.

Adrenal Metastasectomy in the Era of Immuno- and Targeted Therapy.

Adrenal metastasectomy has an increasing role in multimodality oncologic care for diverse primary cancer types. In this review, we discuss the epidemiology, evaluation, and contemporary best practices in the management of adrenal metastases from various primaries. Initial evaluation of suspected adrenal metastases should include diagnostic imaging to assess the extent of tumor involvement and determine surgical resectability, as well as biochemical evaluation for hormone secretion. Biopsy has a minimal role and should only be performed in tumors that are established to be non-hormone secreting and when the biopsy results would change clinical management. Adrenal metastasectomy is associated with survival benefit in selected patients. We suggest that adrenal metastasectomy has the greatest benefit in four clinical scenarios: (1) disease limited to the adrenal gland in which adrenalectomy renders the patient disease-free; (2) isolated progression in the adrenal gland in the setting of otherwise controlled metastatic extra-adrenal disease; (3) need for palliation of symptoms related to adrenal metastases; or (4) in the context of tissue-based clinical trials. Both minimally invasive and open adrenalectomy techniques are safe and appear to have equivalent oncologic outcomes. Minimally invasive approaches are favored when technically feasible while maintaining oncologic principles. A multidisciplinary evaluation including clinicians with expertise in the primary cancer type is essential to the successful management of adrenal metastases.

Adoption of Robotic Adrenalectomy: A Two-Institution Study of Surgeon Learning Curve.

BACKGROUND: Robotic adrenalectomy is feasible and safe, yet concerns over increased operative times and the learning curve (LC) for proficiency have limited its adoption. This study aimed to assess the LC for robotic adrenalectomy. METHODS: This is a two-institution retrospective review of consecutive unilateral minimally invasive adrenalectomies performed by four high-volume adrenal surgeons between 2007 and 2022. Two surgeons transitioned from laparoscopic to robotic adrenalectomy, and two surgeons adopted the approach, with proctoring, after completion of fellowship training without robotic experience. Operative time and complications were analyzed. Multivariable regression was used to identify factors associated with operative time. The number of cases required to overcome the LC was determined using the LC-cumulative-sum (LC-CUSUM) analysis. RESULTS: Of 457 adrenalectomies, 182 (40%) were laparoscopic and 275 (60%) robotic. The robotic approach was associated with shorter median operative time (106 vs 119 min; p = 0.002), fewer complications (6% vs 13%; p = 0.018), and fewer conversions to open adrenalectomy (1% vs 4%; p = 0.030), with no difference between the senior and junior surgeons. On adjusted analysis, factors associated with increased operative time were male sex (p < 0.001), BMI > 30 kg/m2 (p < 0.001), and higher gland weight (p < 0.001). The LC-CUSUM analysis showed proficiency after 8-29 procedures. Compared with the first 10 cases, there was a mean reduction in operative time of 14 min after 10-20 cases, 28 min after 20-30 cases, and 29 min after > 30 cases, regardless of surgeon experience. DISCUSSION: With dedicated teams and proctoring, robotic adrenalectomy can be safely adopted at high-volume centers with a minimal LC.

Engineering functional 3-dimensional patient-derived endocrine organoids for broad multiplatform applications.

BACKGROUND: Recent advancements in 3-dimensional patient-derived organoid models have revolutionized the field of cancer biology. There is an urgent need for development of endocrine tumor organoid models for medullary thyroid carcinoma, adrenocortical carcinoma, papillary thyroid carcinoma, and a spectrum of benign hyperfunctioning parathyroid and adrenal neoplasms. We aimed to engineer functionally intact 3-dimensional endocrine patient-derived organoids to expand the in vitro and translational applications for the advancement of endocrine research. METHODS: Using our recently developed fine needle aspiration-based methodology, we established patient-derived 3-dimensional endocrine organoid models using prospectively collected human papillary thyroid carcinoma (n = 6), medullary thyroid carcinoma (n = 3), adrenocortical carcinoma (n = 3), and parathyroid (n = 5). and adrenal (n = 5) neoplasms. Multiplatform analyses of endocrine patient-derived organoids and applications in oncoimmunology, near-infrared autofluorescence, and radiosensitization studies under 3-dimensional in vitro conditions were performed. RESULTS: We have successfully modeled and analyzed the complex endocrine microenvironment for a spectrum of endocrine neoplasms in 3-dimensional culture. The endocrine patient-derived organoids recapitulated complex tumor microenvironment of endocrine neoplasms morphologically and functionally and maintained cytokine production and near-infrared autofluorescence properties. CONCLUSION: Our novel engineered endocrine patient-derived organoid models of thyroid, parathyroid and adrenal neoplasms represent an exciting and elegant alternative to current limited 2-dimensional systems and afford future broad multiplatform in vitro and translational applications, including in endocrine oncoimmunology.

The use of an educational time-out in thyroid and parathyroid surgery to move the needle in periprocedural education.

BACKGROUND: As surgical training shifts toward a competency-based paradigm, deliberate practice for procedures must be a point of focus. The purpose of this study was to assess the impact of an educational time-out intervention on educational experience and operative performance in endocrine surgery. METHODS: For 12 months, third-year general surgery residents used the educational time-out to establish an operative step of focus for thyroidectomy and parathyroidectomy procedures. Data were collected using the System for Improving and Measuring Procedural Learning application and post-rotation surveys. The Zwisch scale was used to classify supervision, with meaningful autonomy defined as passive help or supervision only. RESULTS: Eight residents and 3 attending surgeons performed the educational time-out for a total of 211 operations (93% completion rate). At the end of each rotation, there was improvement in the frequency of goal setting. There was strong agreement (90%) that the intervention strengthened the educational experience. For most cases (52%), the residents were rated at active help. Residents performed a median of 3/6 thyroidectomy steps at meaningful autonomy and a median of 2/5 parathyroidectomy steps at meaningful autonomy. Review of the qualitative data revealed that optimal feedback was provided in 46% of cases. CONCLUSION: The educational time-out strengthened educational experiences. Stepwise procedural data revealed the varying levels of supervision that exist within an operation. Broader implementation of this intervention could facilitate competency-based procedural education.

Educational Review: Intraoperative Parathyroid Fluorescence Detection Technology in Thyroid and Parathyroid Surgery.

BACKGROUND: Accurate parathyroid gland (PG) identification is a critical yet challenging component of cervical endocrine procedures. PGs possess strong near-infrared autofluorescence (NIRAF) compared with other tissues in the neck. This property has been harnessed by image- and probe-based near-infrared fluorescence detection systems, which have gained increasing popularity in clinical use for their ability to accurately aid in PG identification in a rapid, noninvasive, and cost-effective manner. All NIRAF technologies, however, cannot differentiate viable from devascularized PGs without the use of contrast enhancement. Here, we aim to provide an overview of the rapid evolution of these technologies and update the surgery community on the most recent advancements in the field. METHODS: A PubMed literature review was performed using the key terms "parathyroid," "near-infrared," and "fluorescence." Recommendations regarding the use of these technologies in clinical practice were developed on the basis of the reviewed literature and in conjunction with expert surgeons' opinions. RESULTS: The use of near-infrared fluorescence detection can be broadly categorized as (1) using parathyroid NIRAF to identify both healthy and diseased PGs, and (2) using contrast-enhanced (i.e., indocyanine green) near-infrared fluorescence to evaluate PG perfusion and viability. Each of these approaches possess unique advantages and disadvantages, and clinical trials are ongoing to better define their utility. CONCLUSIONS: Near-infrared fluorescence detection offers the opportunity to improve our collective ability to identify and preserve PGs intraoperatively. While additional work is needed to propel this technology further, we hope this review will be valuable to the practicing surgeon.

Label-Free Enhancement of Adrenal Gland Visualization Using Near-Infrared Autofluorescence for Surgical Guidance.

BACKGROUND: During adrenalectomy, surgeons have traditionally relied on their subjective visual skills to distinguish adrenal glands (AGs) from retroperitoneal fat and surrounding structures, while ultrasound and exogenous contrast agents have been employed for intraoperative AG visualization, all of which have their limitations. We present a novel label-free approach that uses near-infrared autofluorescence (NIRAF) detection, which demonstrates potential for enhanced intraoperative AG visualization and efficient tumor resection during adrenalectomies. METHODS: Patients undergoing adrenalectomy or nephrectomy were enrolled for this feasibility study. NIRAF emitted beyond 800 nm was detected in vivo from AGs and surrounding tissues during open adrenalectomies or nephrectomies. NIRAF was also measured ex vivo in excised AGs following robotic adrenalectomies. NIRAF images of tissues were captured using near-infrared (NIR) camera systems, whereas NIRAF intensities were recorded concurrently using fiber-optic probe-based NIR devices. Normalized NIRAF intensities (expressed as mean ± standard error) were analyzed and compared. RESULTS: Among the 55 enrolled patients, NIRAF intensity was elevated significantly for AGs versus retroperitoneal fat and other structures. NIR images of AGs also revealed a distinct demarcation of NIRAF between adrenal cortex and other periadrenal structures. NIRAF intensity in AGs was decreased markedly in malignant adrenal tumors, while benign adrenal cortical tumors and healthy adrenal cortex exhibited the strongest NIRAF levels. CONCLUSIONS: Our preliminary findings indicate that NIRAF detection could be a promising label-free technology to enhance intraoperative AG visualization and holds immense potential for effective tumor demarcation during cortical-sparing adrenalectomies or adrenal-conserving surgeries.

Cytoreductive Surgery of the Primary Tumor in Metastatic Adrenocortical Carcinoma: Impact on Patients' Survival.

CONTEXT: The role of cytoreduction of adrenocortical carcinoma (ACC) remains poorly understood. OBJECTIVE: To analyze the impact of cytoreductive surgery of the primary tumor in patients with metastatic ACC. DESIGN AND SETTING: We performed a multicentric, retrospective paired cohort study comparing the overall survival (OS) in patients with metastatic ACC who were treated either with cytoreductive surgery (CR group) or without cytoreductive surgery (no-CR group) of the primary tumor. Data were retrieved from 9 referral centers in the American-Australian-Asian Adrenal Alliance collaborative research group. PATIENTS: Patients aged ≥18 years with metastatic ACC at initial presentation who were treated between January 1, 1995, and May 31, 2019. INTERVENTION: Performance (or not) of cytoreductive surgery of the primary tumor. MAIN OUTCOME AND MEASURES: A propensity score match was done using age and the number of organs with metastasis (≤2 or >2). The main outcome was OS, determined from the date of diagnosis until death or until last follow-up for living patients. RESULTS: Of 339 patients pooled, 239 were paired and included: 128 in the CR group and 111 in the no-CR group. The mean follow-up was 67 months. Patients in the no-CR group had greater risk of death than did patients in the CR group (hazard ratio [HR] = 3.18; 95% CI, 2.34-4.32). Independent predictors of survival included age (HR = 1.02; 95% CI, 1.00-1.03), hormone excess (HR = 2.56; 95% CI, 1.66-3.92), and local metastasis therapy (HR = 0.41; 95% CI, 0.47-0.65). CONCLUSION: Cytoreductive surgery of the primary tumor in patients with metastatic ACC is associated with prolonged survival.

Initial clinical experiences using the intraoperative probe-based parathyroid autofluorescence identification system-PTeye™ during thyroid and parathyroid procedures.

BACKGROUND AND OBJECTIVE: The Food and Drug Administration has cleared a probe-based near-infrared autofluorescence (NIRAF) detection system called PTeye™ as an adjunct tool for label-free intraoperative parathyroid gland (PG) identification. Since PTeye™ has been investigated only in a "blinded" manner to date, this study describes the preliminary impressions of PTeye™ when used by surgeons without being blinded to the device output. METHODS: Patients undergoing thyroid and parathyroid procedures were prospectively recruited. Target tissues were intraoperatively assessed with PTeye™. The surgeon's confidence in PG identification was recorded concomitantly with NIRAF parameters that were output in real-time from PTeye™. RESULTS: A retrospective review of prospectively collected data on 83 patients was performed. PTeye™ was used for interrogating 336 target tissues in 46 parathyroid and 37 thyroid procedures. PTeye™ yielded an overall accuracy of 94.3% with a positive predictive value of 93.0% and a negative predictive value of 100%. An increase in confidence for intraoperative PG identification with PTeye™ was observed by all three participating high-volume surgeons, irrespective of their level of accrued surgical experience. CONCLUSIONS: Probe-based NIRAF detection with PTeye™ can be a valuable adjunct device to intraoperatively identify PGs for surgeons of varied training and experience.

Patient Perspectives on the Extent of Surgery and Radioactive Iodine Treatment for Low-Risk Differentiated Thyroid Cancer.

OBJECTIVE: To understand patient perspective regarding recommended changes in the 2015 American Thyroid Association (ATA) guidelines. Specifically, in regard to active surveillance (AS) of some small differentiated thyroid cancer (DTC), performance of less extensive surgery for low-risk DTC, and more selective administration of radioactive iodine (RAI). METHODS: An online survey was disseminated to thyroid cancer patient advocacy organizations and members of the ATA to distribute to the patients. Data were collected on demographic and treatment information, and patient experience with DTC. Patients were asked "what if" scenarios on core topics, including AS, extent of surgery, and indications for RAI. RESULTS: Survey responses were analyzed from 1546 patients with DTC: 1478 (96%) had a total thyroidectomy, and 1167 (76%) underwent RAI. If there was no change in the overall cancer outcome, 606 (39%) of respondents would have considered lobectomy over total thyroidectomy, 536 (35%) would have opted for AS, and 638 (41%) would have chosen to forego RAI. Moreover, (774/1217) 64% of respondents wanted more time with their clinicians when making decisions about the extent of surgery. A total of 621/1167 of patients experienced significant side effects with RAI, and 351/1167 of patients felt that the risks of treatment were not well explained. 1237/1546 (80%) of patients felt that AS would not be overly burdensome, and quality of life was the main reason cited for choosing AS. CONCLUSION: Patient perspective regarding choice in the management of low-risk DTC varies widely, and a large proportion of DTC patients would change aspects of their care if oncologic outcomes were equivalent.

Cumulative GRAS Score as a Predictor of Survival After Resection for Adrenocortical Carcinoma: Analysis From the U.S. Adrenocortical Carcinoma Database.

BACKGROUND: Adrenocortical carcinoma (ACC) is a rare but aggressive malignancy, and many prognostic factors that influence survival remain undefined. Individually, the GRAS (Grade, Resection status, Age, and Symptoms of hormone hypersecretion) parameters have demonstrated their prognostic value in ACC. This study aimed to assess the value of a cumulative GRAS score as a prognostic indicator after ACC resection. METHODS: A retrospective cohort study of adult patients who underwent surgical resection for ACC between 1993 and 2014 was performed using the United States Adrenocortical Carcinoma Group (US-ACCG) database. A sum GRAS score was calculated for each patient by adding one point each when the criteria were met for tumor grade (Weiss criteria ≥ 3 or Ki67 ≥ 20%), resection status (micro- or macroscopically positive margin), age (≥ 50 years), and preoperative symptoms of hormone hypersecretion (present). Overall survival (OS) and disease-free survival (DFS) by cumulative GRAS score were analyzed by the Kaplan-Meier method and log-rank test. RESULTS: Of the 265 patients in the US-ACCG database, 243 (92%) had sufficient data available to calculate a cumulative GRAS score and were included in this analysis. The 265 patients comprised 23 patients (10%) with a GRAS of 0, 52 patients (21%) with a GRAS of 1, 92 patients (38%) with a GRAS of 2, 63 patients (26%) with a GRAS of 3, and 13 patients (5%) with a GRAS of 4. An increasing GRAS score was associated with shortened OS (p < 0.01) and DFS (p < 0.01) after index resection. CONCLUSION: In this retrospective analysis, the cumulative GRAS score effectively stratified OS and DFS after index resection for ACC. Further prospective analysis is required to validate the cumulative GRAS score as a prognostic indicator for clinical use.

Practice patterns for surgical management of low-risk papillary thyroid cancer from 2014 to 2019: A CESQIP analysis.

BACKGROUND: Patients with low-risk-PTC who undergo thyroid lobectomy (TL) have comparable disease-specific survival with lower morbidity than total thyroidectomy (TT). We aim to describe the surgical management of low-risk-PTC using the Collaborative Endocrine Surgery Quality Improvement Program (CESQIP). METHOD: CESQIP thyroidectomies of PTC tumors <4 cm were analyzed from 2014 to 2019 (n = 740). Postoperative outcomes were compared. Subgroup analysis examined temporal and institutional trends, and stratified for tumor size. Statistics utilized t-test, ANOVA, and Chi-squared. RESULTS: TT patients had greater hypoparathyroidism, operative time, and length-of-stay (all p < 0.001). Incidence of TL decreased with increasing tumor size (24.2% for <1 cm, 15.8% for 1-2 cm, 6.1% for 2-4 cm). TL rates increased from 2.0% in 2014 to 21.2% in 2018-19. Completion thyroidectomy was recommended in 12.0% of TL subjects. There was significant variation in TL rate by institution (p < .001). CONCLUSIONS: For low-risk-PTC, TT remained the most commonly utilized operation. TL rates increased following release of the new ATA guidelines. TT was associated with higher perioperative morbidity. Further insight is needed to understand factors influencing operative approach.